A model for predicting the performance of IP videoconferencing

This paper investigates the interaction effect of audio andvideo, and studies lip synchronization (lip sync). The studyshows a comprehensive evaluation of achievable audio andvideo quality undertaken based upon different sets ofimpairments between audw and video, prior to transmission.The tests have been conducted on two different taskscenarios, i.e. passive communication and interactivecommunication (person to person). The research concentrates on quantifying the effects of network impairments(packet loss) on perceived audio and video quality, as wellas finding the correlations between audio and video inmultimedia applications. The results presented in this papershow the strong interaction dependency between audio andvideo. It was justified that video has a unique benefit onmultimedia quality for its psychological effects. The findingsalso concluded that the sensory interactions, and theattention given to a particular aspect of performance, areclearly content-dependent Electronic accessThe Emerald Research Register for this joumal is available atht tp: / /vwvw.emeraldinsight .comyresearchregisterThe current issue and full text archive of this joumal isavailable ath t tp : / /wvwv.emeraldins ight .com/1065-0741.htm Campus Wide Information SystemsVolume 20 Number 4 2003 pp. 159-166r MCB UP Limited ISSN 1065̂ 741DDI 10.1108/106S0740310491333IntroductionT h e a i m o f the paper is to investigate thein teract ion efTeci be tween the perceived audioand video quali ty ' i n m u l t i m e d i a services. T h estudy o n l ip sync is also descr ibed i n this paper.L i p sync refers t o the synch ron iza t ion betweenthe movements o f the speaker's l i p s and thespoken voice. L i p sync is one o f the impor tan tissues i n m u l t i m e d i a appl ica t ions .Previous research has c l a imed t h a t a user'sassessment o f a u d i o q u a l i t y is i n f luenced by thepresence o f v ideo i n m u l t i m e d i a appl icat ions(Watson and Sasse, 1996) . Fo r th i s reason, theexperiments were based o n invest igat ing andq u a n t i f y i n g the po ten t i a l i n t e r ac t i on effectbetween audio a n d v ideo w h e n the transportmechanism ca r ry ing the t w o medias is subjectto packet loss.T h e impor t ance o f g o o d qua l i t y audio i n aconference cannot be overstated (Kawalek ,1995; K i t a w a k i and N a g a b u c h i , 1998) . Sincetrue lip reading is imposs ib le fo r most people,effective c o m m u n i c a t i o n cannot be achievedw i t h o u t in te l l ig ib le aud io . I Jkewise , audio delaycan make interact ive c o m m u n i c a t i o n d i f f i c u l t .Also , audio that is not synchron ized w i t h videocan be d is t rac t ing due to loss o f l i psynchroniza t ion .( Airrent desktop v ideoconfe renc ing systemstransmit between t w o and eight f rames o f videoper second (quar ter c o m m o n interchangefo rma t , Q C I F 176 x 144 p ixe l s / commoninterchange f o r m a t , C I F 352 x 2 8 8 pixels),w i t h poor r e so lu t ion and unsynchronized audioand video. ' H i e presence o f v ideo w h i c h enablesin ter j iersonal faceto-facc c o m m u n i c a t i o n isprevalent and m u c h p re fe r red over all humanmeans o f i n t e rac t ion ( T a n g and Issacs, 1993).Studies show tha t , i n workp l ace settings, evenwhen people are given a choice betweend i f fe ren t means o f c o m m u n i c a t i o n , such asemai l , phone and face-to-face, they still chooseface-to-face meet ings f o r p l a n n i n g andde f in i t i ona l tasks ( F i n h o l t et a/., 1990) . Th i s isevidence that v i d e o c o n f e n c i n g has uniquebenefits over a u d i o on ly c o m m m u n i c a t i o n fo rmost classes o f task.M a n y studies have invest igated the influencethat video m e d i a t i o n has o n the process ofc o m m u n i c a t i o n . Some research f indings claimthat the presence o f a v ideo channel does not159 The effects of audio and video correlalion and lip synchronizatianL Mued, B. lines, S. Fumel! and P. ReimJds di rec t ly i m p r o v e the task per formance in thecon tex t o f desktop v ideoconferencing ( D V C )( W i l s o n and Sasse^ 2000a) . However , i t hasbeen suggested that the m a i n use o f the videol i n k i n D V C is psychological ( H a r d m a n ei a/.,1998) such as to c la r i fy meaning, to p rov ide ameans o f c o m m o n reference, to check whetheranyone was speaking d u r i n g an unusual ly longsilence, to give psychological reassurance t l i a tthe other pan ic ipan t s were actually there b ycreat ing a sense o f presence etc. T h u s , i t isstated t h a t j i n general , video is better than aud iof o r i n t e r r u p t j o n s , naturalness, interact ivi ty ,feedback and a t t en t ion (Sellen, 1992).I n s u m m a r y , wh i l s t good qua l i ty video isbenef ic ia l to enhance many interactive tasks,su f f i c i en t aud io qua l i ty is an essential fo rrea l t ime in t e r ac t i on . T h e quest ion is, whatqua l i t y is good enough to meet end user'srequirements?T o date, there is no standard consensus toc l a r i f y m u l t i m e d i a qual i ty o f service (QoS) , I nc o n j u n c t i o n , effect ive evaluation methods arev i ta l to de te rmine tlie q u a l i t j ' the users need losuccessfully p e r f o r m tasks in videoconferences.H o w e v e r , i t is stated that assessing die qual i tyo f aud io and video over ihe IP ne twork offers agreat challenge due to its constantly changingand unpredic tab le nature (Wi l son and Sasse,2000a) . O n the other hand , to determinem u l t i m e d i a conferenc ing qua l i ty has certaind i f f i c u l t i e s , as there is no recognized indust ryconsensus o f wha t really determines audio andv ideo qua l i t y . A t present, i t is o f t e n quest ionedwhe the r the qua l i ty o f the audio and video inm u l t i m e d i a conferenc ing is adequate to carr>' itstask pe r fo rmance ( W i l s o n and Sasse, 2000b) .M a n y researchers c la im that d i f f e ren t tasksp e r f o r m e d by the end user w i l l require d i f f e r en tlevels o f audio and video qual i ty . I n some casesi t may be necessary to pr ior i t ize video overaud io , or vice versa, depending o n the type o fsession. F o r example, language teaching i n adistance learn ing appl ica t ion w i l l require bet teraud io , as opposed to a remote interview thatdemands a good qual i ty o f video as wel l .T h e r e f o r e , i t is essential to investigate whatqua l i t y is necessary f o r each specificapp l i ca t ion . T h e a im o f this research is toestablish t a x o n o m y o f realt ime m u l t i m e d i a taskand appl ica t ions , and to determine theCampus-Wide Information SystemsVotume 20 Number 4 • 2003 1S9-166m a x i m u m and m i n i m u m audio a n d v ideoqua l i ty boundar ies f o r the g iven tasks. The experimentsT h e t w o ma in exper iments descr ibed in thispaper are, f i r s t , E x p e r i m e n t A : investigatein terac t ion effects between perceived qua l i ty o faud io and video and second, E x p e r i m e n t B :study the effects o f l i p sync o n m u l t i m e d i aqual i ty .Experiment A: investigate interactioneffects between perceived quality' of audioand videoAs previously stated, the exper iments werebased u p o n invest igat ing a po ten t i a l in te rac t ioneffect between aud io and v ideo m e d i a i n D V Csystems in the presence o f packet loss. T h eapproach is to send the aud io and v i d e o .componen t w i t l i respect to the assigned qua l i tyf o r each media , i n t w o d i f f e r e n t taskperformances ( i .e . interact ive and passiveinteract ions) . T h e proposed m e t h o d w i l l be todegrade the qua l i ty o f audio and to upgrade thequa l i ty o f v ideo , o r vice versa, before sending i tt h rough a "connect ionless" n e t w o r k . A t thereceiving end, d ie subjects w i l l evaluatei n d i v i d u a l qua l i ty o f audio , v ideo a n d c o m b i n e daudiovisual o f l o w b i t rate v ideoconfe renc ing .F igure I depicts the V o I P (voice over I P ) testbed c o n f i g u r a t i o n used fo r the exper iments , andthe various elements i l lus t ra ted are describedbe low.Tcntiitml A and BT w o ident ical v ideoconfe renc ing systems(hardware and sof tware ) , r u n n i n g M i c r o s o f tN e t M e e t i n g , placed in t w o separate rooms, tobe used b y the subjects to rate m e a n o p i n i o nscores o f the perceived audio and v ideo qual i ty .C P U s : 2 0 0 M H z P e n d u m processors, 6 4 M BR A M , were used. T h e Q C I F 176 x 144 pixels Figure 1 Test bed configuration I STenninal (A)IP.SubnetIP>ubDet 2> Router (C)(NlSTNet)Tciminal (B)